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I.  EXECUTIVE  SUMMARY 


The  DARPA  Airborne  Video  Surveillance  (AVS)  program  was  established  to  develop 
and  promote  technologies  to  make  airborne  video  more  useful,  providing  capabilities  that 
achieve  a  UAV  force  multiplier.  Harris  contributed  to  that  goal  with  the  development  of 
processing  solutions  which  permit  precision  video  georegistration  of  streaming  video 
data.  The  objective  of  this  video  georegistration  technology  development  is  to  add 
operational  value  to  video  surveillance  data  by  (1)  automatically  relating  mission  video 
imagery  to  precision  controlled  reference  imagery  to  achieve  metric-level  targeting 
accuracy  in  near  real  time,  and  (2)  producing  video  products  and  targeting  readouts  from 
streaming  video  data  in  support  of  time  critical  targeting  and  command  and  control 
tasks.  Our  PVR  (Precision  Video  Registration)  solution  substantially  alters  the  current 
mission  model  for  video  interpretation  by  creating  timely,  geospatially  corrected  tactical 
imagery  ready  for  analysis  and  exploitation. 

In  GFY’Ol,  Harris  was  re-directed  to  stop  customer-funded  development  work  on  its  own 
processing  capabilities,  and  tasked  with  validating  the  performance  of  both  Harris  and 
Samoff  Precision  Video  Registration  systems.  Section  It  documents  the  results  of  that 
work.  The  Harris  PVR  registration  performance  results,  relative  to  DOQ  and  DTED 
reference  data,  are  summarized  below  in  Figure  1.1.  Section  IE  provides  a  design 
overview  of  the  Harris  PVR  system,  and  documents  operational  transition  opportunities 
and  associated  technical  issues.  Our  final  activity  summary  may  be  found  in  Section  IV. 
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Figure  1.1:  Harris  EO  Video  to  DOQ  &  DTED  Registration  Results 
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II.  PRECISION  VIDEO  REGISTRATION  VALIDATION 


PVR  Validation  Protocol 


In  1999  and  2000,  the  AVS  program  collected  airborne  video  with  embedded  telemetry 
data  over  4  CONUS  sites  using  a  DeHavilland  DHC-6  Twin  Otter  testbed  aircraft 
maintained  and  operated  by  the  U.S.  Army’s  Night  Vision  and  Electronic  Systems 
Directorate.  The  New  York  Site  contains  hilly  terrain  and  had  snow  cover  when  the  data 
was  collected  in  February  2000.  The  North  Carolina  Site,  a  relatively  flat  littoral  area 
with  a  mix  of  urban,  suburban,  and  rural  land  use,  was  collected  in  March  2000.  The 
desert  valleys  and  rugged  mountains  of  the  Nevada  Site  were  collected  in  May  2000.  We 
also  leveraged  data  collected  in  October  1999  over  the  gently  rolling  terrain  of  the 
Virginia  Site’s  drop  zone  used  for  our  live  integrated  AVS  Flight  Experiment.  The  goal 
was  to  evaluate  PVR  performance  over  a  variety  of  operationally  significant  flight  paths, 
look  angles,  ground  sample  distances,  scene  content  qualities,  seasonal  variations,  and 
terrain  types.  Towards  that  end,  approximately  60  hours  of  EO  and  IR  airborne  video 
data  was  collected  and  distributed  by  the  program  team. 

The  mission  video  data  distribution  format  for  the  AVS  program  is  S-VHS  video  cassette 
tapes.  The  PVR  TSRD  (Technology  Subsystem  R&D)  contractors  are  responsible  for 
their  own  NTSC  analog  video  capture  hardware  within  their  processing  solutions.  The 
associated  metadata  (a.k.a.  exploitation  support  data  (ESD)  or  telemetry  data)  is 
extracted  from  the  VBI  (vertical  blanking  interval)  of  the  NTSC  video  signal  by  Norpak 
TTX745  data  recovery  hardware,  and  the  resulting  serial  data  stream  is  parsed  into 
metadata  packets  by  custom  software  according  to  the  AVS  Common  Air  /  Ground 
System  (CAGS)  Interface  Control  Document  (ICD)  established  at  the  program  Critical 
Design  Review  in  February  1999.  Video  rate  is  nominally  30Hz,  and  the  metadata 
packets  contain  GPS  vehicle  position  updates  at  a  lOHz  rate,  INS  vehicle  attitude 
updates  at  30Hz,  sensor  gimbals  pointing  data  at  25Hz,  and  sensor  field  of  view  updates 
at  6.25Hz. 

PVR  effectively  transfers  the  higher  accuracy  of  controlled  reference  imagery  and 
elevation  data  (NIMA  standard  products)  to  more  recent  mission  video  collected  by 
tactical  airborne  assets  in  order  to  derive  precise  locations  of  entities  and  events  in 
support  of  intelligence,  surveillance,  and  reconnaissance  missions.  USGS  Digital  Ortho 
Quarter  Quad  (DOQ)  Im  GSD  imagery  was  used  by  both  TSRD  contractors  as  an 
unclassified  reference  surrogate  for  the  classified  NIMA  Controlled  Image  Base  (CIB) 
product  in  order  to  facilitate  algorithm  development  within  their  respective  labs.  FOUO 
NIMA  Digital  Terrain  Elevation  Data  (DTED)  is  used  in  conjunction  with  DOQ  or  CIB 
reference  imagery  by  the  3D  processing  algorithms  of  both  PVR  systems.  Since 
classified  Digital  Point  Positioning  Data  Base  (DPPDB)  stereo  imagery  is  the  only 
NIMA  product  certified  for  generation  of  preeision  targeting  coordinates,  we  also 
employed  this  reference  source  in  our  validation  exercise. 
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There  are  many  factors  that  can  influence  the  performance  of  video  georegistration 
algorithms,  generally  falling  into  the  3  classes  of  image  quality,  terrain,  and  viewing 
geometry.  Not  all  of  the  factors  are  independent,  and  the  relationships  and  interactions 
between  them  are  complex  and  nonlinear.  The  result  is  a  large  search  space  that  could  be 
explored  to  characterize  the  performance  envelope  of  our  PVR  systems.  Since  a  brute 
force  attack  on  this  problem  was  economically  unfeasible  and  technically  unjustifiable, 
we  decided  to  focus  on  four  primary  factors:  GSD  (ground  sample  distance),  look  angle 
obliquity,  scene  content  (i.e.,  the  presence  and  distribution  of  distinctive  pattern  structure 
in  the  scene),  and  video-reference  differences  (e.g.,  season,  weather,  time  of  day,  feature 
content  and  shape  changes,  sensor  difference,  etc.).  The  choice  of  these  particular  four 
factors  was  based  on  Harris’  extensive  experience  in  developing  and  evaluating  state  of 
the  art  georegistration  capabilities  for  operational  national  and  tactical  systems. 

Of  the  60+  hours  of  available  video.  Table  2.1  shows  the  17  video  clips  that  were  chosen 
by  Harris  for  their  internal  PVR  validation  experiments.  Each  clip  has  been  assigned  a 
unique  ID  number,  and  is  unambiguously  specified  by  the  site,  mission  tape  ID, 
collection  date,  and  VITC  format  (HHMMSSFF:  hours,  minutes,  seconds,  frame  number) 
start  and  stop  times.  The  validation  protocol  established  by  the  A  VS  program  team  in 
April  2000  defines  a  “standard  clip”  that  is  2  minutes  in  duration,  sub-sampled  at  a 
0.5Hz  frame  rate,  and  manually  truthed  using  the  ME'P'^  Geopositioning  Tool  and 
Accuracy  Assessment  Tool.  These  two  tools  are  supported  by  NIMA  and  used  by  them 
to  generate  targeting  products  and  evaluate  the  accuracy  of  their  sensors  over  time.  The 
rationale  for  the  standard  clip  definition  is  that  it  should  be  sufficiently  long  to  capture 
an  operationally  significant  imaging  CONORS,  balanced  against  the  labor  involved  in 
manually  generating  truth  data  for  performance  characterization.  Having  60  truthed 
frames  uniformly  distributed  throughout  the  2  minute  clip  was  deemed  reasonable.  The 
Frames  column  of  Table  2.1  indicates  the  number  of  frames  the  human  operator  was 
actually  able  to  mensurate  out  of  the  total  number  captured  with  approximately  2  seconds 
between  them. 
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The  clip  mensuration  process  involves  the  operator  displaying  each  of  the  60  video 
frames,  orthorectified  based  on  their  raw  telemetry  data.  The  operator  identifies  a 
minimum  of  6  salient  points  per  video  frame  that  are  also  readily  identifiable  in  the 
reference  imagery.  The  Geopositioning  Tool  is  used  to  compute  the  ground  point 
location  (latitude,  longitude,  height)  of  an  image  point  using  the  reference  data  (either 
DOQ  and  DTED,  or  a  DPPDB  stereo  pair).  This  point  is  then  transferred  to  the 
Accuracy  Assessment  Tool  so  that  the  operator  can  specify  the  corresponding  image 
point  in  the  video  frame.  The  manually  dropped  points  are  used  as  truth  data.  The  3D 
Euclidian  distance  between  the  unregistered  image  points  is  measured,  and  the  average 
for  a  frame  is  used  to  characterize  its  raw  ESD  error  (also  known  as  the  initial  alignment 
error). 

The  image  coordinates  (row,  column)  of  the  truth  points  were  distributed  to  the  TSRD 
contractors,  along  with  the  corresponding  digitized  video  frames  and  raw  ESD,  so  that 
their  PVR  algorithms  could  register  the  images  and  provide  reported  ground  coordinates 
for  the  truth  points.  These  reported  ground  coordinate  locations  for  the  same  salient 
feature  points  were  used  to  compute  residual  alignment  error  for  the  respective  PVR 
systems  in  the  same  manner  that  initial  alignment  error  was  computed  (averaging  over  all 
the  truth  points  to  arrive  at  a  measurement  for  each  frame).  The  Harris  PVR  system  was 
able  to  process  the  60  mensurated  frames  per  “standard  clip”  to  produce  its  results.  The 
Samoff  PVR  system,  which  has  a  hardware  front-end  component,  required  provision  of 
an  additional  540  frames  per  clip,  captured  by  the  same  digitizer,  in  order  to  process  at  a 
5Hz  frame  rate. 

PVR  Performance  Results 


In  this  section  we  present  PVR  performance  results  for  the  10  trials  indicated  by  the 
legend  of  Table  2.1,  6  trials  using  DOQ  imagery  and  DTED  as  reference  data,  and  4 
trials  using  DPPDB  as  reference  data.  For  each  clip  we  have  plotted  initial  ESD 
alignment  error,  as  well  as  residual  alignment  error  after  processing  by  the  Harris  and 
Samoff  Precision  Video  Registration  systems. 

DOQ  &  DTED  Reference  Data 

For  our  first  6  validation  trials,  we  chose  2  clips  from  each  of  the  collection  sites  for 
which  we  had  unclassified  DOQ  reference  imagery  available  (New  York,  North  Carolina, 
and  Virginia).  For  the  Virginia  Site,  the  AVS  program  was  able  to  obtain  DTED  Level  3 
(10m  post  spacing)  elevation  data,  while  only  DTED  Level  1  (100m  post  spacing)  was 
available  for  the  New  York  and  North  Carolina  Sites.  We  show  the  reference  imagery  by 
itself  in  panel  (a)  of  each  of  the  clip  summary  figures  (Figures  2.1  through  2.6),  while 
panel  (b)  shows  a  mosaic  of  the  video  frames,  with  later  frames  on  top  of  older  frames, 
overlaying  the  reference  DOQ  image.  Panel  (c)  plots  the  alignment  errors  for  each 
mensurated  frame  in  the  clip,  with  clip  performance  sununary  statistics  tabulated  below 
(median,  90*  percentile,  mean,  standard  deviation,  and  percentage  of  frames  meeting 
successful  registration  criterion  of  10m  or  better  residual  alignment  error). 
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As  shown  in  Figure  2.1,  Clip  A2  for  the  New  York  Site  was  collected  with  the  sensor 
pointing  down  and  directly  ahead  while  flying  a  long  linear  run  heading  southwest  across 
a  river  valley.  The  seasonal  differences  (ice  covered  river  and  snow  covered  ground)  are 
clearly  evident.  For  New  York  Clip  A4,  shown  in  Figure  2.2,  the  sensor  stares  at  the 
intersection  of  two  roads  and  a  railroad  as  the  aircraft  circles  it.  In  addition  to  the  snow 
cover,  note  the  large  stand  of  trees  northwest  of  the  intersection  that  have  been  cleared 
between  the  mission  and  reference  data  collection  times.  Even  with  these  video¬ 
reference  differences,  both  PVR  systems  produced  excellent  results  for  the  New  York 
Site. 

Circle  stare  and  long  linear  run  imaging  CONOPS  were  also  chosen  for  the  North 
Carolina  Site  clips.  Although  the  data  formats  and  validation  protocols  were  the  same  as 
they  were  for  the  New  York  site,  the  North  Carolina  Site  proved  to  be  problematic  for  the 
Samoff  PVR  system.  They  were  unable  to  process  Clip  B3  at  all,  and  so  panel  (c)  in 
Figure  2.3  presents  only  the  Harris  PVR  results  plot  and  summary  statistics.  The 
“before”  data  series  in  that  panel  characterizes  the  initial  ESD  alignment  error,  the  “kf  ’ 
series  is  the  residual  alignment  error,  and  “type”  is  a  status  line  that  indicates  the  state  of 
the  Harris  Dynamic  Video  Worm  algorithm,  described  in  Section  HI  of  this  Final 
Technical  Report.  Clip  B4  of  Figure  2.4  contains  a  long  linear  run  flying  south  over 
varied  scene  content.  In  particular,  note  the  cleared  vs.  built-up  differences  in  the 
suburban  development  at  the  top  (north  end)  of  the  run,  challenging  both  PVR  systems  at 
the  beginning  of  the  clip. 

The  collections  at  the  Virginia  Site  focused  on  the  live  processing  vignettes  for  the 
integrated  flight  experiments  and  demonstrations  held  in  the  fall  of  1999.  Therefore,  all 
PVR  clips  for  this  site  focus  on  the  same  drop  zone  area,  and  are  differentiated  primarily 
by  date.  For  Clip  D2  of  Figure  2.5,  the  aircraft  flew  a  U-shaped  pattern,  with  the  first  leg 
starting  north  of  the  drop  zone  and  heading  southeast.  The  depression  angle  of  the 
sensor  was  45°,  with  an  azimuth  angle  of  approximately  45°  to  the  right  of  the  aircraft 
heading.  The  sensor  stared  at  the  building  compound  at  the  southeast  end  of  the  drop 
zone  as  the  aircraft  did  a  clockwise  turn  about  it,  straightening  out  into  the  last 
northwestward  leg  of  the  U  south  of  the  drop  zone  with  the  sensor  depression  and 
azimuth  angles  both  at  45°  as  before.  The  Samoff  PVR  system  again  had  problems  with 
this  clip,  dropping  the  last  10  frames  and  biasing  their  summary  statistics  presented  in 
panel  (c)  of  Figure  2.5.  The  sensor  pointing  angles  for  Clip  D4  of  Figure  2.6  were  both 
held  fixed  at  45°  as  in  D2,  with  the  aircraft  heading  east  while  flying  north  of  the  drop 
zone,  then  turning  southeast  to  scan  its  length.  The  poor  scene  content  (mostly  a  dense 
forest  of  trees)  at  the  beginning  of  the  clip  contributed  to  both  PVR  systems  having  a 
difficult  time  getting  started. 

A  summary  of  both  TSRD’s  EO  video  registration  to  DOQ  &  DTED  reference  data 
processing  results  is  presented  in  Figure  2.7. 
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Figure  2.1:  NY-A2  Deferiet  Zamboni  Clip 
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Figure  2.2:  NY-A4  Intersection  Circle  Stare  Clip 
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Figure  2.4:  NC-B4  Suburban  Run  Clip 
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Figure  2.5:  VA-D2  15-Oct  Fast  Straight  Line  Clip 
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Figure  2.6:  VA-D4  19-Oct  Straight  Line  Clip 
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Figure  2.7:  PVR  System  Validation  for  DOQ  &  DTED 
DPPDB  Reference  Data 

For  our  last  4  validation  trials,  we  chose  one  clip  for  each  of  the  4  collection  sites  to 
register  against  DPPDB  stereo  reference  imagery.  Because  complete  DOQ  processing 
results  were  expected  but  never  received  from  Sarnoff,  they  were  never  sent  the  selected 
DPPDB  clips  for  processing.  Since  the  reference  imagery  is  classified,  we  show  only  the 
mosaicked  video  frames  comprising  the  clips  in  panel  (a)  of  Figures  2.8  through  2.1 1 . 
Note,  however,  the  large  difference  in  collection  times  between  the  video  and  reference 
imagery  documented  in  these  figures,  which  proved  to  be  a  major  factor  in  the  Harris 
performance  results  presented  in  panel  (b)  of  the  figures. 

Clip  A3  from  the  New  York  Site  is  a  circle  stare  about  a  large  industrial  chimney  located 
at  the  center  of  the  scene,  as  shown  in  Figure  2.8.  The  highly  oblique  sensor  look  angle, 
combined  with  the  seasonal  variation  and  significant  age  difference,  made  this  a 
particularly  challenging  clip.  The  North  Carolina  Clip  B1  comprised  half  of  a  U-shaped 
pattern  (one  leg  and  a  turn),  the  first  portion  of  which  was  exceedingly  difficult  because 
the  causeway  was  tom  down  and  replaced  with  a  new  one  170m  to  the  west  of  its 
original  location  in  the  DPPDB  imagery  (moved  from  the  easternmost  north/south  road 
in  the  video  mosaic  to  the  location  shown  in  Figure  2.9).  The  Nevada  Clip  C5  is  a  long, 
linear  mn  headed  south  with  the  sensor  looking  ahead  with  a  45°  depression  angle,  as 
shown  in  Figure  2.10.  The  scene  content  was  reasonable,  consisting  of  desert  arroyos, 
but  was  made  difficult  by  high  contrast  shadows  of  high  altitude  clouds  moving  quickly 
across  the  scene.  The  final  clip,  D4  from  the  Virginia  Site  in  Figure  2.1 1,  is  the  same 
one  used  in  the  DOQ  validation  trials  to  provide  a  means  of  comparison  between  the 
reference  types.  A  summary  of  Harris  PVR  registration  to  DPPDB  is  shown  in  Figure 
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Figure  2.10:  NV-C5  Desert  1.5m  GSD  Run  Clip 
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Figure  2.11:  VA-D4  19-Oct  Straight  Line  Clip 
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Figure  2.12:  Harris  Summary  DPPDB  Registration  Performance 


PVR  Validation  Conclusions 


In  general,  the  performance  of  both  PVR  systems  met  the  A  VS  program  goal  of 
providing  better  than  10  meters  CE  registration  accuracy,  relative  to  the  reference  data,  at 
a  1.0  Hz  frame  processing  rate  or  better.  Clearly,  there  is  no  significant  accuracy 
difference  in  the  performance  of  the  validated  versions  of  the  two  PVR  systems  we 
examined.  Absolute  error  characterization  was  not  possible  due  to  the  program’s  failure 
to  collect  and  provide  adequate  site  survey  data  required  to  perform  that  task  (the  NG- 
CMS  provided  data  was  examined  and  determined  to  be  not  suitable).  Against  DOQ  and 
DIED  reference  data,  the  success  rate  was  consistently  greater  than  90%  using  this 
criterion,  with  an  overall  mean  of  3 -5m  of  residual  alignment  error.  Against  DPPDB 
stereo  reference  imagery,  Harris  showed  a  success  rate  of  60%,  with  the  mean  relative 
residual  alignment  error  at  approximately  the  10m  success  threshold  due  to  the  age  of  the 
reference  and  the  difficulty  of  the  data  sets. 
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III.  FINAL  PVR  DESIGN  HIGHLIGHTS 


At  customer  direction,  there  has  been  no  DARPA-sponsored  development  of  the  Harris 
Precision  Video  Registration  (PVR)  system  since  the  September  2000  AVS  Program 
Management  Review  at  Harris.  This  section  provides  highlights  of  the  current  Harris 
Precision  Video  Processing  capabilities,  which  have  been  maintained,  enhanced, 
extended,  and  ported  to  new  platforms  under  Internal  Research  and  Development 
funding.  We  also  address  operational  transition  opportunities  and  technical  issues. 

Harris  PVR  Design 


The  current  Harris  PVR  system  architecture,  shown  in  Figure  3.1,  is  designed  to 
accommodate  the  real-time  flow  of  video  images  and  telemetry  support  data  into  the  four 
AVS  PVR  CSCIs  (Computer  Software  Configuration  Items).  This  architecture  produces 
an  asynchronous  flow  of  improved  telemetry  to  PVR  subscribers  and  also  supports  ad 
hoc  requests  for  georeferenced  orthomosaics,  precision  geolocation  results,  and  improved 
telemetry  for  specified  video  frames.  The  architecture  utilizes  multiple  processes  that 
share  telemetry  data  via  a  dynamically  updating  database  residing  in  shared  memory. 
Processes  within  PVR  communicate  via  PVM  (Parallel  Virtual  Machine)  messages.  The 
PVR  Services  API  wraps  a  CORE  A  interface  to  provide  Precision  Telemetry  Broadcast 
packets  and  Mosaic  Update  imagery  patches  to  the  HCI,  AM,  and  MTS  within  the  SGI 
Infinite  Reality  hardware  and  AVS  CAGS  (Common  Air  /  Ground  System)  software 
architectures. 
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Figure  3.1:  Harris  PVR  System  Architecture 
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Figure  3.2:  VET  Live  Video  Displays 


[Easting:  285923.56  ■  Northing:  3845121.64  m  Elev:  12.1  m 


Figure  3.3:  VET  Exploitation  Displays 
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Real  Time  Video  Registration  Processing 


Figure  3.4:  Real  Time  Video  Registration  (RTVR)  Architecture 


The  live  video,  targeting,  and  situational  awareness  displays  developed  for  the  Harris 
Video  Exploitation  Toolkit  (VET)  COTS  package  (shown  in  Figures  3. 2-3. 3),  and  a 
Georegistered  Video  Database  (GVDB)  prototype  of  a  motion  imagery  archive  /  GIS 
system  are  also  licensable  PVR  client  capabilities  developed  under  Harris  IR&D  funding. 

The  Telemetry  Processes  CSCI  of  Figure  3.1  contains  the  RTVR  kernel,  the  heart  of  the 
PVR  technology.  Figure  3.4  (Real  Time  Video  Registration  Architecture)  illustrates  the 
key  components  of  the  current  design.  Previous  Quarterly  Scientific  and  Technical 
Reports  provided  complete  processing  details  for  the  GFY’99-GFY’00  versions  of 
RTVR.  Since  that  time,  Harris  has  developed  a  proprietary  Dynamic  Video  Worm 
registration  algorithm  that  robustly  accommodates  varying  scene  content  in  narrow  field 
of  view  airborne  video  imagery.  A  prototype  version  of  this  algorithm  was  validated  in 
GFY’Ol  (see  Section  n  for  details)  and  presented  at  the  first  IEEE  Workshop  on  Video 
Registration,  held  in  conjunction  with  the  Eighth  IEEE  International  Conference  on 
Computer  Vision  in  July  2001 . 

RTVR  registers  incoming  video  image  frames  to  reference  imagery  and  provides  near- 
real-time  (NRT)  adjustments  to  the  telemetry  stream,  thereby  providing  improved 
accuracy  for  video  users  (HCI  operators  and/or  embedded  CAGS  applications).  Key 
features  of  the  architecture  include: 

•  Inputs: 

•  Raw  telemetry 
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•  Video  frames  (IR  or  EO) 

•  Requests  for  telemetry  at  frame  times  with  desired  quality-of-service  level 

•  Processing: 

•  An  autonomous  frame  selection  process 

•  Interpolation  of  metadata  (due  to  various  acquisition  rates) 

•  NRT  multi-stage  match  point  generation  between  incoming  video  frames  and 
reference  imagery 

•  Prescreening  of  the  match  point  sets  to  determine  the  success  or  failure  of  the 
current  mission-to-reference  frame  registrations,  which  is  used  to  control  the  set 
of  frames  participating  in  “video  worm”  and  generation  of  frame-to-frame 
matches 

•  A  new  least  squares  pre-adjustment  of  the  bundle  of  frames  comprising  the 
“video  worm”,  which  are  then  fed  sequentially  into  the  Kalman  filter  adjustment 
stage 

•  Adjustment  of  the  telemetry  stream  via  a  Sequential  State  Estimator  (Kalman 
filter  which  estimates  the  state  of  a  system  of  measurements  containing  random 
errors) 

•  Outputs: 

•  “Precision  Broadcast”  product  -  adjusted  telemetry  for  automatically  selected 
frames 

•  “Telemetry  Request”  products;  four  levels  of  service  to  provide  adjusted 
telemetry  in  response  to  ad  hoc  requests  for  specific  frames 

•  Imaging  Conops 

•  PVR  uses  single  sensor  #1  (EO  or  IR)  in  wide  to  medium  FOV.  PVR  registration 
performance  is  related  to  the  accuracy  and  resolution  of  the  reference  imagery 
and  terrain  digital  elevation  model.  Very  narrow  FOV  operations  may  require 
high  resolution  reference  imagery,  and  are  automatically  accommodated  by  the 
Dynamic  Video  Worm  processing  algorithm. 

•  Key  Challenges 

•  Large  Engineering  Support  Data  (ESD)  errors  with  broad  variances 

•  Narrow  Field  of  View  (NFV)  (scene  content  matching  difficulty  with  small 
comparison  patches) 

•  Shallow  sensor  elevation  angles 

•  Sensor/video  irregularities 


At  the  August  2001  AVS  Final  Capabilities  Demonstration,  Harris  showed  the  complete 
PVR  System  Architecture  of  Figure  3.1  performing  real  time  end-to-end  processing  of 
S-VHS  tapes  of  Twin  Otter  mission  data  collected  for  the  PVR  Validation  task,  just  as  it 
would  perform  live  processing  if  integrated  into  the  AVS  CAGS  as  in  the  October  1999 
Flight  Experiment.  Using  an  SGI  Origin  3200,  we  demonstrated  a  0.98  Hz  frame 
registration  rate  with  a  5.1  second  latency.  The  PVR  System  software  configuration 
consisted  of  the  PVR  Manager,  5  Mosaic  Service  processes,  the  RTVR  Controller,  5 
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Correspondence  Generators,  and  the  Worm  Combiner  application.  The  Origin  3200 
hardware  configuration  consisted  of  eight  400MHz  R 12000  CPU’s  with  SMB  of  cache 
per  CPU,  4GB  of  RAM,  and  six  36GB  JBOD  (just  a  bunch  of  disks)  with  Fiber  Channel 
I/O.  An  SGI  02  handled  Video  Capture,  HMI  Client  Windows,  and  the  Telemetry  Ingest 
application. 


PVR  Technology  Transition  Opportunities  and  Issues 

We  also  worked  towards  a  GFY’Ol  AVS  goal  of  demonstrating  PVR  technology  on  RQ- 
1  Predator  data.  Issues  with  the  existing  Predator  ISR  system  include  raw  ground  error 
on  the  order  of  hundreds  of  meters  to  several  kilometers,  unsynchronized  video  and 
metadata  streams,  and  a  very  low  0.3  Hz  metadata  sampling  and  distribution  rate.  We 
obtained  the  Predator  Metadata  System  ICD  and  developed  ingest  software  compliant 
with  that  API.  Modifications  required  to  readily  accommodate  this  format  of  metadata 
were  made  to  our  PVR  system.  We  obtained  an  EEG  Model  DE241DR  hardware  data 
recovery  decoder  in  order  to  process  the  current  Predator  data  distribution  format,  and 
upgraded  an  existing  Norpak  Model  TTX745  data  recovery  decoder’s  firmware  in 
anticipation  of  a  planned  Predator  Metadata  System  upgrade  to  achieve  a  higher  20  Hz 
metadata  encoding  rate. 

We  developed  a  rigorous  and  a  simplified  Predator  sensor  model,  both  of  which  are 
currently  being  calibrated  with  data  collected  in  December  2001  (see  Figure  3.5). 
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Figure  3.5:  December  2001  Predator  Data  Collection 

Harris  has  identified  and  pursued  two  follow-on  opportunities  with  NMA.  In 
preparation  for  the  NIMA-MIPO  sponsored  Motion  Imagery  Precision  Engagement 
Program,  we  have  been  internally  funding  a  port  of  our  prototype  PVR  processing 
capability  to  the  Harris  ORIGIN  COTS  product  software  baseline  to  leverage  a  new, 
state-of-the-art  registration  processing  architecture.  This  effort  also  facilitates  hosting 
our  new  VET  product  under  the  Windows  NT  operating  system,  a  requirement  for 
delivering  the  capability  to  the  NIMA  motion  imagery  exploitation  cells  and  the  Predator 
Ground  Control  System  (GCS).  A  manual  bootstrap  HMI  has  been  designed  to 
accommodate  the  large  ground  error  and  low  distribution  rate  of  the  current  Predator 
metadata  stream.  Harris  has  also  identified  and  is  pursuing  a  migration  path  that  will 
quickly  lead  to  compliance  with  the  NMA  Motion  Imagery  Standard  Board’s 
requirements  and  recommendations  for  standard  data  formats  and  interfaces. 

For  the  NMA  Tactical  Smart  Image  ACTD,  we  will  build  on  our  existing  PVR  video-to- 
DPPDB  registration  capability  by  incorporating  our  fully  autonomous  solution  to  DEM 
extraction  from  stereo  reference  imagery.  We  will  integrate  a  new  replacement  sensor 
model  and  error  propagation  formulation  developed  by  NMA  to  autonomously  produce 
a  new  “smart  image”  product  from  streaming  video  that  is  suitable  for  precision  target 
coordinate  generation.  We  will  deliver  this  capability  to  MIPO  and  the  Predator  GCS  at 
Indian  Springs,  both  of  which  require  Windows  NT  and/or  Windows  2000  support.  In 
order  to  meet  additional  planned  deliveries  to  CAOC-X  and  the  DCGS  sites,  we  must 
address  security  issues  related  to  joint  and  coalition  forces’  centers  of  operation,  as  well 
as  achieve  DII  COE  Level  5  compliance.  It  is  expected  that  NMA  will  help  fund  these 
compliance  issues,  in  addition  to  supporting  certification  of  the  geospatial  accuracy  of 
our  motion  imagery  processing  capabilities. 
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IV.  FINAL  ACTIVITY  SUMMARY 


Accomplishments  for  Quarter  Ending  July  1998 
R&D  Trades: 

♦  Completed  first  cut  on  the  Sequential  State  Estimator  (SSE) 

♦  Identified  the  metadata  inputs,  system  parameters,  and  level  system  calibration  and 
characterization,  necessary  to  the  development  of  the  sensor  error  model 

♦  Developed  video  mosaic  architecture  and  thread  diagrams 

PVR  Design  Development 

♦  Created  first  draft  of  the  PVR  system  architecture,  threads  and  timing  budgets 

♦  Identified  the  CAGS  Air  Ground  system  partitions  for  the  PVR 

♦  Identified  system  interface  requirements 

♦  Identified  99  PVR-CAGS  architecture  performance  criteria  and  timing  budgets 

♦  Identified  00  PVR-CAGS  architecture  performance  criteria  and  timing  budgets 

♦  Identified  PVR-CAGS  computations  and  storage  budgets 

♦  Identified  and  assessed  video  storage  and  control  COTS  applications 

♦  Identified  PVR-CAGS  MetaData  types 

♦  Developed  a  PVR  Glossary  of  Terms  and  Acronym  list 

♦  Provided  a  Test  Plan  for  99  and  00  architectures 

♦  Provided  a  staged  Deliver  Plan 

Documentation 

♦  Provided  a  Pre  kickoff  packages  on  the  PVR  design  and  scope 

♦  Provided  review  and  inputs  for  the  June  Data  Collection  Plan. 

♦  Provided  first  draft  of  the  HLDD/CONOPS  before  Kickoff 

♦  Provided  PVR  system  architecture,  threads,  and  timing  budgets  and  collaborated  with 
SI  on  the  CAGS  environment 

♦  Provided  inputs  for  the  August  Data  Collection  Plan. 


Workshops 

♦  Presented  the  HLDD  content  at  Kickoff  at  the  AVS-SI  [CMI]  installation  in 
Maryland. 

♦  Supported  the  system  architecture  working  groups  at  the  kickoff 

♦  Completed  all  Action  Items  assigned  to  Harris  in  Pre-Kickoff 

♦  Completed  of  all  Action  Items  assigned  to  Harris  at  the  Kickoff 

♦  Established  Motion  Imagery  Group  lab  at  Harris 

♦  Arranged  for  HLDD  review  meeting  with  CMI  at  Harris  on  July  13-14th. 
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System  Demonstration  98 

♦  Developed  first  draft  of  the  ‘98  Demonstration  Plan  and  a  schedule  to  provide 
demonstrations  at  Harris  and  support  PDR  “bazaar”  in  September. 

Emerging  Work 

♦  Began  the  design  audit  of  the  Registration  Architecture  and  considerations  for 
prototyping 

♦  Began  the  development  of  a  CONORS  for  Mosaics 

♦  Began  development  trades  for  the  Sequential  State  Estimator 

♦  Began  the  assessment  of  the  June  Data  Collection 

Accomplishments  for  Quarter  Ending  November  30***  1998 

R&D  Trades: 

♦  Prepared  for  utilizing  the  Twin  Otter  sensor  by  becoming  more  familiar  with  the 
Islander  Video  Image  and  Telemetry  data  set.  The  following  are  lessons  learned  from 
the  Islander  sensor  suite  imagery  and  telemetry: 

•  We  found  the  images  to  have  significant  smearing  from  mechanical 
modulations  due  to  the  location  of  the  sensor 

•  Attempted  to  get  a  good  calibration  run  on  ground  objects. 

•  The  calibration  runs  have  shown  the  need  for  good  DTED  (possibly  Level  3). 

♦  Defmitized  the  draft  criteria  for  frame  selection 

♦  Began  taking  statistics  on  Islander  Telemetry  to  quantify  the  sub-packet  rates 
necessary  on  the  Twin  Otter. 

♦  Provided  a  CONORS  for  Mosiacs  paper 

♦  Performed  simulation  experiments  validating  the  utility  of  flow  vector  information 
within  a  sequential  state  estimator  framework  in  order  to  improve  registration 
performance. 

PVR  Design  Development 

♦  Created  first  draft  of  the  PVR  Software  Design  Document 

♦  Created  a  draft  PVR  ICD 

♦  Completed  a  ‘99  PVR  Schedule  for  Design,  Development,  and  Integration,  and  a 
staged  delivery  plan 

♦  Updated  the  HLDD  to  include  the  PVR  architecture  and  timelines 

♦  Completed  the  AVS-PVR  Preliminary  Design  Review 

♦  Received  a  ROM  from  Earth  Data  to  produce  a  3”  and  6”  per  pixel  set  of 
orthorectified  reference  images  of  the  Ft.  A.P.  Hill  area. 

♦  Collaborated  with  CMI  on  the  CAGS  API  development 

Documentation 

♦  Provided  copies  of  the  draft  updates  of  the  HLDD  to  CMI 

♦  Provided  a  draft  of  the  PVR  API  (ICD) 


AVS-PVR  FINAL  TECH  REPORT  Version  2/22/02 


Page  28 


CDRL  EOOl 


Doc  Number:  AVS-PVR  Final 


♦  Developed  object-oriented  design  documentation  within  the  Rational  Rose  tool  in 
preparation  for  detailed  design  reviews  of  the  PVR  Manager,  Image  Control,  and 
Telemetry  process  CPCI’s. 

Workshops 

♦  Presented  the  PDR  of  the  system  architecture  interfaces  and  technical  performance 
measures  and  a  Development  and  Integration  Plan 

System  Demonstration  98 

♦  Demonstrated  the  Harris  registration  of  video  images  at  the  PDR  and  MSTAR 
Program  Demonstration  in  September. 

Accomplishments  for  Quarter  Ending  February  28. 1999 

PVR  Design  Research  and  Development 

♦  Prepared  and  presented  to  DARPA  end-users  at  the  AVS  CONOPS  Working  Group 

♦  Completed  internal  Registration  Algorithm  Design  Review 

♦  Completed  coding  &  unit  testing  of  the  internal  framework  of  multiple  threaded 
processes 

♦  Finished  the  detailed  design  and  coding  of  initial  versions  of  each  CSCI  (software 
component)  of  the  PVR  architecture 

♦  Worked  with  TSRDs  and  CMI  to  develop  a  system  level  imaging  CONOPS  (Concept 
of  Operations)  to  focus  the  system  design  through  determination  of  how  the  sensors 
would  be  used  in  ’99 

♦  Collaborated  with  CMI  on  the  CAGS  API  development 

♦  Prepared  and  presented  PVR  architecture  at  the  pre-CDR  Software  Working  Group 

♦  Completed  the  AVS  Critical  Design  Review 

♦  Completed  two  additional  Technical  Reviews  with  DARPA 

♦  Developed  &  executed  detailed  experiments  to  characterize  &  assess  the  Islander  data 
&  RTVR  design  (ongoing  -  see  “RTVR  Development  and  Experimentation”  below) 

♦  Developed  and  began  experimentation  on  a  solution  for  a  new  requirement:  PVR  to 
generate  flow  vectors  without  MTS  hardware  inputs  (ongoing  -  see  “RTVR 
Development  and  Experimentation”  below) 

♦  Received  VBI  decoder  hardware,  software  and  test  videotapes  with  VBI  encoding, 
and  began  developing  the  software  to  extract  Twin  Otter  VBI  metadata 

♦  Pursued  and  received  partial  information  on  Twin  Otter  sensor/gimbal/airframe 
metadata  (detailed  ICD-type  information)  and  error  characterization 

♦  Collaborated  with  MTS  to  develop  a  joint  MTS/PVR  Vignette 

♦  Coordinated  NIMA  and  NVESD  on  acquisition  of  DTED  level  2-3 

♦  Pursued  additional  precision  reference  imagery  for  Ft.  A.P.  Hill  via  commercial 
sources. 
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Documentation 

♦  Prepared  the  PVR  Detailed  Design  Document 

♦  Prepared  the  PVR  Critical  Design  Review  package 

♦  Prepared  two  additional  detailed  technical  briefings  for  DARPA 

♦  Updated  the  CAGS  API  Document 

♦  Updated  the  High  Level  Design  Document  (HLDD)  to  reflect  the  current  design  and 
alternatives  being  researched/developed 

♦  Updated  the  Software  Development  Plan  (SDP) 

♦  Prepared  the  Mar.  ’99  Twin  Otter  Collection  Plan 

♦  Updated  the  PVR  Software  Design  Document  and  object-oriented  design 
documentation  within  the  Rational  Rose  toolset 

RTVR  Development  and  Experimentation 

♦  This  was  the  focus  of  key  research  and  technology  efforts  for  the  quarter.  The 
February  1999  Quarterly  Scientific  &  Technical  Report  contains  detailed  status. 

Accomplishments  for  the  Quarter  Ending  May  31. 1999 

PVR  Design  Research  and  Development 

♦  Completed  two  additional  Technical  Reviews  with  DARPA 

♦  Performed  Islander  data  experiments  to  validate  both  Sequential  State  Estimators 
(Match  Point  Generator  SSE  and  Global  Flow  SSE)  on  real  Islander  video  data 

♦  Prototyped  an  implementation  to  incorporate  scale  and  rotation  effects  into  our  match 
point  consistent  subset  criterion  and  reduced  the  execution  time  of  this  algorithm 

♦  Tracked  results  of  affine/projective  transform  experiments  on  our  Video  IR&D 
(assessed  potential  leverage  of  this  work  for  PVR  match  point  generation). 

♦  Completed  experiments  on  a  solution  for  a  new  requirement:  PVR  software-based 
generation  of  flow  vectors  without  MTS  hardware  inputs 

♦  Developed  and  tested  a  working  metadata  decoder  and  a  video-ingest  capability  to 
extract  Twin  Otter  VBI  metadata  and  allow  time-based  look-up  of  video  frames  using 
VBI  decoder  hardware  and  videotapes  with  VBI  encoding 

♦  Pursued,  received  and  incorporated  into  the  design  more  information  on  Twin  Otter 
sensor/gimbal/airframe  metadata  (detailed  ICD-type  information)  and  error 
characterization 

♦  Selected  and  extracted  video  and  telemetry  clips  from  the  March  99  Twin  Otter  data, 
characterized  and  plotted  Otter  ESD,  found  the  data  quality  and  telemetry  is  superior 
to  Islander  data 

♦  Discovered  significant  scaling  effects  (3%  to  39%  error)  due  to  incorrect  linear 
mapping  from  zoom  factor  to  FOV  in  CAGS  metadata  computations.  Compensated 
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for  this  in  the  current  data  set,  and  designed/scheduled  experiments  to  determine  a 
complete  calibration  table  for  the  program  in  the  May  data  collection 

♦  Gimbal  azimuth  and  elevation  angle  errors  were  found  to  be  an  order  of  magnitude 
greater  than  the  original  spec,  resulting  in  the  addition  of  new  adjustable  parameters 
to  our  sensor  model 

♦  Conducted  initial  validation  of  RTVR  design  approach  using  the  March  Twin  Otter 
data;  pulled  202  frames  of  varying  (best  and  worst)  registration  difficulty  (depression 
angle  and  scene  content)  and  achieved  99.5%  success  rate  (accuracy  improved),  with 
86%  of  the  trials  meeting  the  goal  of  <  10  meter  error. 

♦  Analyzed  timing  issues  associated  with  metadata  delivery  and  frame-to-frame 
stabilization  parameters,  created  a  block  diagram  of  the  Twin  Otter  in  the  process. 

♦  Demonstrated  a  "loose"  end-to-end  integration  which  proved  we  can  ingest  video  and 
telemetry  from  our  CAGS  API  emulators,  and  build  a  mosaic  based  on  raw 
(uncorrected)  telemetry  in  real  time 

♦  Evaluated  different  versions  of  SVHS  tape  dubs  from  the  May  99  Twin  Otter  EO 
collection;  the  tape  received  from  a  NVESD  dubbing  consultant  on  5/21  proved  to 
have  the  smallest  metadata  dropout  rate  of  0.14  frames/second  dropout  rate  (or  4.7  x 
10'^  error  rate),  and  image  quality  noticeably  better  than  the  original  tape  dub 
received  on  5/12 

♦  Developed  a  new  level  of  “adjusted  ESD”  service  that  eliminates  RTVR  dependence 
on  MTS  Stabilization  Parameters  for  user-specified  frames 

♦  Continued  upgrade  of  our  SGI  02's  and  our  SGI  Octane  server  to  IRIX6.5.3  and  the 
7.2.1  compiler,  and  conducted  integration/builds  with  the  software 

♦  Delivered  118  PTRs  this  quarter  of  new  code  and  changes  that  arose  out  of  our  end- 
to-end  integration  of  our  CPCIs;  all  CPCIs  except  the  RTVR  kernel  and  PVR/C  AGS 
integration  classes  were  completed;  the  size  of  the  AVS0.03  baseline  at  the  end  of  the 
May  period  is  14477  LSS  of  new,  136000  LSS  of  reused/re-engineered  code 

♦  Continued  Coordination  with  NIMA,  DARPA  and  NVESD  on  acquisition  of  DTED 
level  2-3,  helped  obtain  firm  order  for  DTED  3  (projected  delivery  July  99) 

♦  Continued  to  pursue  additional  precision  reference  imagery  for  Ft.  A.P.  Hill  via 
NIMA  and/or  commercial  sources 

Documentation 

♦  Prepared  two  additional  detailed  technical  briefings  for  DARPA 

♦  Updated  the  CAGS  API  Document 

♦  Updated  the  High  Level  Design  Document  (HLDD)  to  reflect  design  ehanges 

♦  Prepared  the  May  ’99  Twin  Otter  Colleetion  Plan,  and  alternative  plans  to  work 
around  dual  sensor  collection  difficulties 

♦  Wrote  and  delivered  the  Quarterly  Management  Report  (delivered  3/1 1) 

♦  Wrote  and  delivered  the  Quarterly  Scientific  and  Technical  Report  (delivered  3/17) 

♦  Provided  updates  to  the  A  VS  Integration  Milestones  by  Month  document  and 
schedule  to  CMI 
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RTVR  Development  and  Experimentation 

♦  This  was  the  focus  of  key  research  and  technology  efforts  for  the  quarter.  The  May 
1999  Quarterly  Scientific  &  Technical  Report  contains  detailed  status. 

Accomplishments  for  the  Quarter  Ending  September  30. 1999 

PVR  Development  and  Integration 

June  1999: 

♦  For  the  first  time,  demonstrated  end-to-end  Real  Time  Video  Registration  in  the 
Harris  lab  running  on  multiple  SGI  02’ s  and  an  Octane.  Multiple  processes 
communicating  via  PVM  ingested  video  and  telemetry  and  registered  streaming 
video  frames  as  fast  as  possible  and  logged  the  results  in  a  Telemetry  Database  for  as 
long  as  we  cared  to  mn  the  system  (10-15  minutes),  with  excellent  results. 

♦  Ported  the  AVS-PVR  software  baseline  to  IRIX6.5.3  and  the  7.2.1  compiler  to  match 
the  SDL  development  environment. 

♦  Supported  the  June  In-Process  Working  Group  at  Belcamp,  MD. 

July  1999: 

♦  Supported  an  AVS  Technical  Review  for  the  DARPA  SPO  director. 

♦  Delivered  PVR  Release  1 .0  compiled  against  the  first  available  GAGS  Ground  Build 
that  met  the  CDR  spec  (CORBA-free  API).  All  PVR  API  interfaces  are  available  for 
link  testing  in  the  SIL. 

♦  DTED  Level  3  reference  data  received  and  utilized  in  experiments  characterizing 
RTVR  processing  results  vs.  DTED  Level  1. 

August  1999: 

♦  Shipped  PVR  Release  2.0,  which  included  the  Affine  Consistent  Subset  RTVR 
algorithm  implementation  with  quantitative  processing  results  for  March  PVR  data. 

♦  Supported  the  August  In-Process  Review  at  Ft.  A.  P.  Hill. 

♦  Shipped  PVR  Release  3.0,  which  consisted  primarily  of  integration  testing  fixes  and 
the  Telemetry  Rate  implementation. 

September  1999: 

♦  Performed  “reported  field  of  view”  to  “adjusted  focal  length”  calibration  of  the  AVS 
EO  sensor  model  based  on  new  PVR  data  collected  on  8/18  and  8/24.  Implemented 
calibration  of  the  AVS  FLIR  sensor  model  based  on  experimental  measurements 
made  by  the  Army  Night  Vision  Lab  in  their  Advanced  Sensor  Evaluation  Facility. 

♦  Implemented  the  Mono  Match  Point  leg  of  the  RTVR  kernel  to  increase  PVR 
robustness. 
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♦  Encountered  metadata  decoder  hardware  problems  with  the  August  and  September 
PVR  tapes,  eventually  traced  cause  to  encoder  gain  changes  within  CAGS. 

♦  Shipped  PVR  Release  4.0  on  9/22,  began  delivering  patched  versions  of  the  PVR 
DSO  library  every  other  day  via  the  ISDN  connection  to  the  SEE  as  integration 
problems  were  found  and  fixed.  Continued  to  work  video  and  telemetry  ingest 
problems  with  the  SI.  SI  begins  HCI  link  testing  against  PVR  API. 

Accomplishments  for  the  Quarter  Ending  December  31. 1999 

PVR  Development  and  Integration 

October  1999: 

♦  Shipped  PVR  Release  5.0  linked  against  final  CAGS  video  &  metadata  ingest  fixes. 

♦  Internal  PVR  integration  completed  during  on-site  testing  at  Ft.  A.  P.  Hill  Oct.  5-8th. 

♦  Delivered  PVR  Release  6.0,  which  included  two  significant  PVR  API  interface 
changes  to  facilitate  CAGS  HCI  integration,  implementation  of  a  scale  change  in  our 
AVS  sensor  model  to  account  for  an  unexpected  CAGS  video  digitizer  sampling  rate, 
and  modification  of  PVR  service  apps  to  save  image  and  geometry  files  in  order  to 
overcome  HCI  deficiencies  via  use  of  the  MET®  tool. 

♦  Supported  on-site  integration  at  Ft.  A.  P.  Hill  Oct.  13-18*,  which  included  3  live 
flight  test  experiments.  Compiled  preliminary  RTVR  processing  results.  CORE  A 
distribution  of  PVR  data  had  problems,  and  the  threaded  PVR  mosaic  processes 
would  not  run  on  the  SGI  Onyx2  Infinite  Reality  the  way  they  ran  in  Harris  lab.  Built 
a  more  complete  Im  GSD  reference  image  of  the  demo  area  from  USGS  DOQ’s. 

♦  Successfully  accomplished  first  major  milestone  of  the  AVS  program  by 
demonstrating  a  Live  Flight  Experiment  at  Ft.  A.  P.  Hill  on  Oct.  19-20*  to  DARPA 
and  their  invited  guests. 

November  1999: 

♦  Continued  RTVR  performance  analysis  and  tuning  activities,  including  experiment- 
driven  fixes  to  the  kernel. 

♦  Conducted  in-house  post-demonstration  assessment  of  PVR  performance  using 
approved  MOEs. 

♦  Supported  DARPA-AVS  promotional  video  shoot,  built  a  new  mosaic  demo  clip. 

♦  Performed  more  extensive  analysis  of  the  effects  of  the  reference  DEM  on  RTVR 
performance  (DTED  Level  1  vs.  USGS  DEM  vs.  DTED  Level  3). 

♦  Documented  PVR  for  GFY’99  in  this  Quarterly  Scientific  and  Technical  Report. 

December  1999: 

♦  Finished  calibration  of  15  October  data  collect,  began  calibration  of  13,  16,  and  19 
October  data  collects  to  finish  out  GFY’99  performance  analysis. 

♦  Began  port  of  AVS2x  baseline  to  IRIX6.5  in  order  to  completely  synchronize  the 
Harris  software  development  environment  with  the  GFY’OO  SIL. 
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♦  Supported  ECP  response  to  the  GFY’OO  RFP  based  on  program  redirection  by 
DARPA  management. 

Accomplishments  for  the  Quarter  Ending  March  31.  2000 
PVR  Development  and  Integration 

The  main  thrust  of  this  past  quarter  was  in  developing  detailed  plans  for  our  PVR 
experimental  evaluation  and  technology  development  activities  for  GFY’OO.  However, 
extensive  experimentation  was  done  in  the  Harris  Lab  to  continue  making  progress. 

We  documented  our  October  8th,  15th,  13th,  16th,  and  19th  processing  results  for  Ft. 
A.P.  Hill  data.  We  also  described  our  N-View  algorithm  experiments  and  results.  Mono 
Match  Point  utility  experiments  and  results,  and  plans  for  experimentally  evaluating  our 
SSE  and  finalizing  the  state  vector  model  it  employs. 

January  2000: 

♦  Continued  calibration,  processing,  and  analysis  of  all  October  ’99  PVR  data  collects. 

♦  Developed  GFY’OO  Technology  Development  Plan  in  response  to  CECOM  and 
DARPA  request  for  program  re-planning  based  on  DARPA  revision  to  the  technical 
management  strategy  for  the  A  VS  program. 

♦  Obtained  Predator  metadata  specs  and  video  data  from  Samoff. 

♦  Obtained  and  reviewed  DPPDB  performance  specs  and  product  classification  guide. 

♦  Continued  port  of  PVR  software  baseline  to  IRIX6.5. 

♦  Prepared  and  delivered  5*  Quarterly  Scientific  and  Technical  Report  CDRL. 

February  2000: 

♦  Submitted  the  Harris  A  VS  RFP  response. 

♦  Completed  initial  N-View  algorithm  experiments,  planned  SSE  evaluation. 

♦  Received  DPPDB  reference  imagery  for  the  GFY’OO  collection  sites  from  the  govt., 
awaiting  DD  Form  254  authorization  to  process  the  data. 

♦  Helped  govt,  procure  DOQ  reference  imagery  for  Ft.  Drum.  Planned  data  collection 
flights,  provided  on-site  support  at  Ft.  Drum  Feb.  22-25*. 

♦  Calibrated  hundreds  of  October  ’99  video  frames,  continued  performance  analysis. 

♦  Completed  port  of  PVR  baseline  to  IRIX6.5. 

March  2000: 

♦  Submitted  two  clarifications  and  modifications  responses  to  DARPA  and  CECOM. 

♦  Submitted  a  detailed  GFY’OO  Experiment  Plan  documenting  our  data  collection 
requirements  and  capability  development  schedule.  Generated  an  AVS  Data 
Management  Plan  describing  the  organization  of  AVS  data  and  archival  scheme  for 
PVR  experiment  and  evaluation  processing  results.  Supported  program-wide 
telecons  and  started  a  draft  GFY’OO  Evaluation  Plan  documenting  how  PVR 
performance  should  be  measured  and  reported. 

♦  Completed  Mono  Match  Point  algorithm  experiments. 
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♦  Planned  Camp  Lejeune  data  collection  flights  for  the  week  of  3/27/00.  Received 
Harris  copies  of  the  Ft.  Drum  video  data  on  3/30/00.  Received  new  Predator  video 
data  from  NIMA  (collected  Sept.  ’99).  Received  DD  Form  254  authorization  for 
DPPDB  late  in  the  month  and  AVS  personnel  were  briefed  on  lab  security 
procedures,  so  we  are  set  up  to  process  DPPDB  imagery  starting  in  April. 

♦  PVR7.0  pro-forma  external  release  ready  for  delivery  on  schedule,  but  shipment 
delayed  until  the  proper  compiler  version  is  available  in  the  SIL  for  on-site 
integration  and  testing. 

Accomplishments  for  the  Quarter  Ending  June  30. 2000 
PVR  Development  and  Integration 

The  main  thrust  of  this  past  quarter  was  in  developing  new  DPPDB  stereo  registration 
capabilities,  continuing  to  refine  our  registration  algorithms,  and  rapidly  producing 
performance  results  against  the  latest  GFY’OO  data  sets.  The  team  has  been  challenged 
by  the  late  start  this  GFY,  which  resulted  in  a  delayed  staffing  ramp-up,  and  the  delays  in 
getting  the  necessary  equipment  for  our  classified  processing.  However,  extensive 
experimentation  was  done  in  the  unclassified  Harris  Lab  to  continue  making  progress. 
Considerable  effort  was  also  expended  on  establishing  a  PVR  Metrics-Based  Evaluation 
Plan  and  Predictive  Performance  Model,  which  is  documented  in  a  detailed  briefing  to  be 
presented  next  quarter. 

April  2000: 

♦  Hosted  on-site  working  meeting  with  DARPA  PM  and  SI  to  discuss  program-wide 
Video  Georegistration  Evaluation  Process  issues  on  4/1 1/2000. 

♦  Visited  NG-CMS  on  4/26  to  brief  them  and  Samoff  on  the  Harris  PVR  video 
processing  tools.  Demonstrated  the  use  of  MET®  to  drop  calibration  points, 
discussed  other  PVR  applications  which  then  measure,  plot,  and  roll  up  statistics  for 
registration  performance  characterization.  The  program  appears  to  be  converging  on 
the  Harris  tools  and  methodology,  but  no  firm  commitment  has  been  made  by  the  SI. 
Followed  direction  of  DARPA  PM  to  focus  on  experimental  characterization  of  PVR 
performance  within  our  lab  using  all  available  data. 

♦  Completed  Mono  Match  Point  experiments,  as  planned.  Continued  to  evaluate  and 
refine  our  Real  Time  Video  Registration  approach.  SSE  (Kalman  filter)  experiments 
are  under  way. 

♦  Delivered  PVR  Release  7.0  to  NG-CMS,  continuing  to  make  our  internal  tools 
available  to  the  program  for  external  evaluation  purposes. 

♦  Re-organized  and  documented  our  data  management  scheme  to  prepare  for  the 
onslaught  of  video  we  have  to  process  this  GFY.  Received  Ft.  Drum  and  Camp 
Lejeune  data,  planned  Fallon  NAS  collect.  Installed  DewDrop  (GOTS  DPPDB 
viewer)  in  the  classified  TSL  facility,  ordered  stereo  display  hardware.  Began 
processing  Ft.  Drum  data  in  the  unclassified  AVS  lab. 

♦  Prepared  and  delivered  6-7*  Quarterly  Scientific  and  Technical  Report  CDRLs. 
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♦  Obtained  a  Department  of  Commerce  export  control  license  waiver  for  the  MET/PVR 
software  on  4/26  in  order  to  facilitate  collaboration  with  the  University  of  Central 
Florida.  Still  awaiting  DARPA  written  approval  to  release  AVS  data  to  UCF. 

May  2000: 

♦  Brought  on  another  SW  engineer  to  help  the  team  quickly  build  visualization  tools 
and  scripts  to  facilitate  data  set  QA,  telemetry  analysis,  reference  imagery 
determination,  video  clip  selection,  and  capture  of  frame  sequences  to  support 
experiments  and  performance  evaluations. 

♦  Ran  telemetry  database  analysis  tool  on  Oct.  ’99  Ft.  A.P.  Hill  data  sets,  diagnosed 
metadata  decoding  HW  sensitivity  to  choice  of  VTR,  use  of  GFE  VTR  obtained  last 
month  fixes  the  associated  problems.  Began  validation  of  Ft.  Drum  and  Camp 
LeJeune  data  sets.  Selected,  ordered,  and  received  reference  DOQ’s  for  Ft.  Drum  and 
Camp  LeJeune.  Began  selecting,  pulling,  and  processing  video  clips. 

♦  Continued  SSE  (Kalman  filter)  experiments.  Performance  is  better  than  expected  for 
hard  registration  events.  Currently  determining  optimal  state  vector  and  error  model. 

♦  Designed  applications  for  pulling  relevant  stereo  pairs  from  DPPDB  tapes. 

Finalizing  preliminary  design  of  reference  image  manager  and  stereo  registration 
approach.  SGI  02  workstations  and  disk  drives  for  classified  TSL  lab  due  to  arrive 
at  the  end  of  the  month  (just  in  time). 

♦  Working  with  Samoff  to  establish  Video  Geolocation  terrain,  imagery,  metadata,  and 
registration  metrics  and  models.  Jointly  preparing  a  briefing  for  AVS  DARPA  PM  to 
present  to  the  DARPA  SPO  Director.  Continuing  to  leverage  our  valuable 
collaboration  with  UCF.  Metrics  is  proving  to  be  a  very  interesting  research  problem. 

♦  Received  written  permission  from  DARPA  PM  to  release  AVS  video  and  telemetry 
data  to  UCF.  Delivered  binary  release  of  AVS  baseline  and  500  calibrated  video 
frames  from  Oct.  ’99  Ft.  A.P.  Hill  data  sets  for  initial  study. 

June  2000: 

♦  Hosted  a  productive  on-site  visit  by  the  DARPA  PM  on  6/6/200.  Decreased  priority 
of  imagery  metrics  in  favor  of  getting  DPPDB  processing  implemented  ASAP. 

♦  The  AVS  SI  finally  decided  on  MET®  as  the  tool  set  they  will  use  to  drop  points  to 
measure  registration  performance  on  the  program.  Sent  them  a  MET  w/  DPPDB  beta 
release  under  a  restricted  use,  demonstration/evaluation  only  license.  Built  additional 
tools  for  working  with  DPPDB  data. 

♦  Nailed  down  the  program-wide  ICD  for  distributing  video  clips  and  evaluating 
registration  performance,  began  implementing  the  ripple  through  our  existing 
processes  and  code.  Planned  development  of  terrain/image/metadata/registration 
metrics  and  a  predictive  PVR  performance  model.  Collaborating  with  Samoff  on  a 
briefing  for  the  Director  of  the  DARPA  SPO  planned  for  July. 

♦  Arranged  7/6/2000  trip  to  General  Atomics  Aeronautical  Systems  for  a  technical 
interchange  meeting  concerning  PVR  value-added  processing  for  the  Predator  UAV. 
Established  contact  with  NIMA  Motion  Imagery  Standards  Board,  planned  to  attend 
MISB  meeting  on  7/27  in  Reston,  VA.  Downloaded  all  available  docs  from  their 
website,  focused  on  their  Metadata  specs  which  we’ll  try  to  influence  to  ensure  our 
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ability  to  add  value  to  the  airborne  video  systems  we’ll  be  integrating  with  in  the 
future.  Supported  five  other  video-related  new  business  pursuits. 

♦  Selected  10  clips  for  the  July  17*  performance  evaluation,  began  pushing  data 
through  the  processing  pipeline.  Progress  has  been  hampered  by:  (1)  AVS  CM/data 
server  down  for  3  weeks  due  to  HW  failure;  (2)  excessive  delays  in  getting  our  SGI 
02  workstations  for  classified  processing  in  the  TSL  (19  weeks  from  order 
submission  until  resources  finally  ready  to  use);  (3)  staffing  transitions  (training  and 
resource  re-allocation  for  5  incoming,  2  outgoing  video  team  members  in  the  last  2 
months);  and  (4)  received  2  copies  of  half  the  Ft.  Drum  DPPDB  instead  of  one  copy 
of  both  halves. 

♦  Arranging  July  meeting  with  Kalman  filter  experts  outside  of  our  immediate  team 
from  Harris  GCSD  and  the  Florida  Institute  of  Technology  to  debate  mathematical 
model  issues  and  finalize  our  Sequential  State  Estimator  design.  Swirling  on  stereo 
registration  and  stereo-based  point  geolocation  algorithms. 

♦  Coordinated  DR&D  support  to  streamline  video  clip  calibration  processing  via 
enhancements  to  Geopositioning  and  Accuracy  Assessment  tools  within  MET. 

Accomplishments  for  the  Quarter  Ending  September  30,  2000 

PVR  Development  and  Integration 

July-August  2000: 

♦  Visited  General  Atomics  Aeronautical  Systems  on  7/6  for  TIM  on  a  PVR  /  Predator 
integrated  demo.  Lots  of  good  info  exchange  with  their  systems  engineers. 

♦  Held  AVS  SSE  Review  on  7/12.  Invited  Prof.  Fred  Ham  of  FIT,  J.  Stiver  of  GCSD 
Palm  Bay,  and  Garnett,  McDowall,  and  Bell  from  the  Harris  DPL  to  get  outsider 
suggestions  and  insight.  Established  an  experimental  close-out  plan  for  our  Kalman 
Filter  work. 

♦  PVR  technology  development  highlights  include: 

-  Stood  up  initial  DPPDB  registration  implementation. 

-  Image  Metrics  computation  designed  (Entropy  &  Mutual  Information). 

-  Developed  pre-screener  algorithm  to  keep  outliers  out  of  Kalman  filter. 

-  Refined  our  Affine  Consistent  Subset  algorithm. 

-  Defined  experiment-driven  development  path  for  Dynamic  N-View  Video 
to  Stereo  Reference  Consistent  Subset  algorithms  &  architecture. 

♦  Processed  6  stressing  clips  from  Ft.  Drum  and  Camp  Lejeune  data  sets.  Selecting 
additional  Lejeune  and  Fallon  NAS  clips  to  process  next.  Coded  applications 
compliant  with  program-wide  ICD  in  preparation  for  8/28  On-Site  Evaluation  of 
PVR  Performance  by  NG-CMS. 

♦  Participated  in  AVS  Program  Review  on  August  10-11. 

-  Briefed  Bob  Hummel,  the  new  DARPA  PM,  on  our  PVR  development  plan  and 
presented  our  promising  preliminary  DPPDB  performance  results. 

-  Will  write  whitepaper  to  secure  our  AVS  GFY’Ol  funding  of  $1.2M  for  (1)  analysis- 
based  registration  extensions,  (2)  technology  transition  activities  (e.g..  Predator  demo), 
and  (3)  digital  video  architecture  studies  (MPEG-2, 480p,  VISP  metadata). 
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-  Hummel  invited  the  program  team  to  submit  "seedling"  ideas  that  could  be 
demonstrated  this  year  and  lead  to  follow-on  programs. 

♦  The  NIMA  Prototype  Facility  -  Reston  (NPF-R)  has  a  new  Video  /  Motion  Imagery 
Office.  They  have  a  live  Predator  feed,  an  Onyx2  Infinite  Reality,  and  lots  of  RAID 
disk  space,  suitable  for  a  strong  PVR  technology  transition  demo. 

♦  Delivered  Quarterly  Scientific  &  Technical  Report  and  Quarterly  Status  & 
Management  Report  CDRLs.  Shipped  an  updated  MET  w/  DPPDB  release  to  NG- 
CMS. 

September  2000; 

♦  PVR  technology  development  highlights  include: 

-  Completed  first-pass  implementation  of  Stereo  Consistent  Subset,  beginning  testing 
on  DPPDB  data  in  the  TSL  classified  processing  faciUty. 

-  Began  coding  Mutual  Information  and  Entropy  metrics  to  quantitatively 
characterize  PVR  input  imagery. 

-  Refined  experiment-driven  development  path  for  Dynamic  N-View  Video  to  Stereo 
Reference  Consistent  Subset  algorithms  &  architecture. 

♦  Calibrated  4  Lejeune  and  2  Fallon  NAS  clips.  Having  difficulty  processing  the 
mountain  of  data  while  simultaneously  attempting  new  algorithm  development. 

♦  The  AVS  SI,  NG-CMS,  was  a  no-show  for  the  scheduled  8/28  On-Site  Evaluation  of 
PVR  Performance. 

♦  Hosted  an  on-site  AVS  Program  Review  for  Bob  Hummel,  the  new  DARPA  PM,  on 
9/14.  Gave  him  a  complete  history  of  AVS-PVR  development,  current  status,  and 
future  plans,  including  in-depth  technical  details  and  lab  demos.  He  has  proposed 
that  Harris  take  on  the  SI  role  for  GFY’Ol  to  ensure  that  the  technology  transition 
activities  are  successful,  and  the  previously  promised  funding  has  been  cut  from 
$1.2M  to  $700K  to  focus  on  this  task. 

Accomplishments  for  the  Quarter  Endin2  December  31. 2000 

PVR  Development  and  Integration 

October  2000: 

♦  PVR  technology  development  highlights  inelude; 

-  Extracted  DTED  from  DPPDB  for  Drum  and  Fallon  sites  in  order  to  facilitate  point 
transfer,  as  required  by  our  new  Stereo  CSS  algorithms.  LeJeune  and  A.P.  Hill  DEM 
extractions  are  in  work. 

-  Finished  coding  Mutual  Information  and  Entropy  metrics  to  quantitatively 
characterize  PVR  input  imagery.  Began  using  these  algorithms  to  diagnose  Stereo  CSS 
behavior  in  the  TSL. 

-  Began  implementing  Dynamic  Video  Worm  registration  algorithm  to  bridge  scene 
content  gaps  and  increase  robustness. 

♦  Pulled  and  began  calibrating  last  2  Fallon  NAS  and  4  A.P.Hill  clips  for  PVR 
performance  characterization  using  DPPDB.  Received  summer  Ft.  Drum  data  on 
10/26  (the  last  data  set  we'll  have  to  process).  All  clips  will  be  processed  and 


AVS-PVR  FINAL  TECH  REPORT  Version  2/22/02 


Page  38 


CDRL  EOOl 


Doc  Number:  AVS-PVR  Final 


performance  results  will  be  rolled  up  once  the  Stereo  CSS  problem  is  diagnosed  and 
fixed  (NLT  11/10). 

♦  Reconfigured  an  AVS  02  workstation  to  support  standalone  demos  of  the  A  VS  HMI 
in  the  field. 

♦  Working  on  the  SOW  and  schedule  for  the  AVS  GFY'02  ECP. 

♦  Prepared  &  presented  a  PVR  overview  paper  at  the  29th  AIPR  Workshop.  Submitted 
a  research  paper  summary  for  the  DCV'Ol  conference,  where  we  hope  to  present  the 
details  of  our  Kalman  filter  study.  We  also  plan  to  present  a  paper  on  our  new  Stereo 
CSS  &  Dynamic  Video  Worm  registration  algorithms  at  the  Workshop  on  Video 
Registration  to  be  held  in  conjunction  with  the  International  Conference  on 
Computer  Vision  (ICCV2001)  in  July  2001. 

♦  Delivered  the  Quarterly  Scientific  &  Technical  Report  CDRL.  This  year's  Computer 
Software  Product  End  Item  CDRL  was  delivered  to  NG-CMS  in  August. 

November  2000: 

♦  All  staff  has  rolled  off  or  greatly  reduced  hours  until  GFY'Ol  funding  arrives. 

♦  Dynamic  Video  Worm  algorithm  implementation  and  testing  should  be  completed 
this  month. 

♦  Approximately  20  evaluation  test  clips  will  be  ready  for  processing  and  analysis  at 
the  end  of  November. 

♦  The  revised  SOW  and  high-level  schedule  for  GFY'Ol  have  been  submitted  for 
customer  approval. 

December  2000: 

♦  All  staff  has  rolled  off  until  GFY'Ol  funding  arrives. 

♦  Supported  an  AVS  Planning  Meeting  held  at  Samoff  Corp.  on  Dec.  13th. 

♦  First  spiral  of  Dynamic  Video  Worm  algorithm  implementation  was  completed. 
Testing  revealed  a  problem  that  could  not  be  resolved  before  AVS  funding  ran  out. 

♦  17  evaluation  test  clips  are  be  ready  for  processing  and  analysis,  but  there  is  currently 
no  program  funding  to  run  on  the  data. 

♦  The  RFP  from  CECOM  arrived  on  December  20th,  our  response  is  due  January  26th. 

Accomplishments  for  the  Quarter  Ending  March  31, 2001 

PVR  Transition  and  Validation 

January  2001: 

♦  All  staff  has  rolled  off  until  GFY'Ol  funding  arrives. 

♦  The  GFY'Ol  SOW  has  been  split  into  2  parts  (in-scope  and  out-of-scope  with  respect 
to  the  existing  contract).  We  submitted  our  response  to  CECOM's  RFP  for  the  in¬ 
scope  portion  on  25  January.  This  should  get  us  the  first  funding  increment  of 
$400K  ASAP.  We  are  expecting  a  second  "out-of-scope"  RFP  for  $300K  to  arrive  in 
the  next  few  weeks. 

♦  Submitted  our  Quarterly  Scientific  &  Technical  Report  CDRL. 
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February  2001: 

♦  First  increment  of  GFY'Ol  funding  ($400K)  arrived  week  of  2/19.  Finalizing  staffing 
plan,  recalling  team  to  kick  off  this  year's  work  starting  week  of  3/5. 

♦  The  GFY'Ol  SOW  has  been  split  into  2  parts  (in-scope  and  out-of-scope  with  respect 
to  the  existing  contract).  We  submitted  our  response  to  CECOM's  RFP  for  the  out- 
of-scope  portion  on  22  February.  This  should  get  us  the  second  funding  increment  of 
$300K  "soon." 

March  2001: 

♦  Kicked  off  this  year's  work  the  week  of  3/5.  First  set  of  tasks  include  getting  the  SW 
baseline  back  in  order,  documenting  the  existing  system,  up-front  systems 
engineering  for  building  the  Predator  model,  and  lead  work  for  the  validation  tasks. 
Still  waiting  for  word  on  the  final  increment  of  GFY'Ol  funding  ($300K)  so  we  can 
begin  validation  work. 

♦  Samoff  is  having  problems  getting  their  Acadia  HW  PVR  solution  up  and  running, 
so  we'll  end  up  evaluating  their  current  lab  solution. 

Accomplishments  for  the  Quarter  Endin£  June  30. 2001 

PVR  Transition  and  Validation 

Apiil-May  2001: 

♦  Hosted  an  AVS  Program  Review  at  the  Harris  Melbourne,  FL  facility  on  May  1st. 

♦  Preparing  for  a  final  AVS  Demonstration  in  Arlington,  VA,  on  July  31st.  Re-planned 
the  program  to  accommodate  this  activity  (not  in  our  2  ECP's).  Negotiating  the 
possibility  of  obtaining  loaner  HW  from  SGI  so  we  can  put  on  the  best  possible  demo 
of  our  PVR  system  for  the  government. 

♦  Coordinating  with  Samoff  on  the  Validation  Task.  They  have  been  less  than 
cooperative  in  helping  us  evaluate  performance  of  both  the  Harris  and  Samoff  PVR 
solutions. 

♦  Established  a  good  rapport  with  General  Atomics,  completed  the  up-front  systems 
engineering  work  necessary  to  build  Predator  sensor  models.  Two  models  will  be 
necessary  to  accommodate  the  various  Predator  configurations,  one  has  been 
implemented  and  the  second  is  almost  complete.  Beginning  to  build  a  telemetry 
decoder,  will  validate  everything  when  we  receive  useful  data. 

♦  Attended  JEFX'02  Industry  Days,  31  May  - 1  June;  promoting  our  Video  Time 
Critical  Targeting  whitepaper,  supported  by  SMC,  in  support  of  tech  transition. 

June  2001: 

♦  The  Dynamic  Video  Worm  registration  algorithm  developed  under  IR&D  funding 
has  proven  to  be  a  real  winner  during  the  AVS  Validation  against  DOQ  reference 
imagery.  Our  best  clip  had  a  mean  of  3.0m  and  std.  dev.  of  1.3m  with  100%  of  the 
frames  yielding  <10m  post-registration  error.  7  of  9  clips  have  better  than  10m  error 
for  at  least  90%  of  the  frames,  and  the  other  two  have  significant  scene  content 
changes  which  spoil  their  results  to  6-7m  mean  &  4m  std  dev,  with  -80%  frames 
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<10m  error.  The  results  we  will  be  presenting  at  the  IEEE  Workshop  on  Video 
Registration  are  superior  to  those  Samoff  will  be  presenting  at  the  main  ICCV  2001 
conference. 

♦  After  attending  JEFX'02  Industry  Day  on  31  May  &  1  June,  we  determined  that  our 
DARPA  prototype  is  too  far  away  from  the  mature  product  state  required  to  meet  all 
of  the  interoperability  and  security  certification  requirements  of  JEFX  under  their 
planned  schedule  (too  many  hoops  and  too  many  complicating  factors,  including 
simultaneous  exercises  for  Millenium  Challenge,  joint  US  and  NATO  coalition 
participation,  etc.).  SMC  (our  sponsor)  did  not  appear  to  be  a  major  player,  and  there 
was  not  enough  time  to  gamer  ASC  and  AC2ISR  support  for  our  PVP  solution. 
However,  we  did  learn  a  great  deal  about  the  current  Time  Critical  Targeting  toolset 
and  Air  Operations  Center  architecture,  as  well  as  identify  good  leads  for  other 
transition  opportunities  and  high-priority  customers  to  brief  on  our  A  VS  successes. 

♦  The  14  June  briefing/TIM  with  NIMA  MIPO  was  very  productive.  That  customer 
enthusiastically  supports  us,  stating  "Harris  has  the  right  product  at  the  right  place  at 
the  right  time  to  dovetail  with  our  efforts."  They  want  to  evaluate  our  correlation 
algorithms  for  JTW,  are  hooking  us  up  with  GAVWG  to  discuss  our  Predator  model 
and  ideas  for  predictive  error  measures,  and  want  us  to  participate  in  a  NIMA  AT 
study  being  conducted  by  Purdue  to  temper  their  analytical  foundation  with  our  real- 
world  airborne  video  experience. 

♦  The  final  AVS  Capabilities  Demonstration  in  Arlington,  VA,  has  been  postponed  to 
August  30th.  We  have  provided  a  Harris  demo  description  to  the  SETA,  and  held  a 
half-day  TIM  with  SGI  on  6/12  to  negotiate  a  loaner  8  processor  Origin  3000  series 
computer  since  the  CAGS  Onyx2  Infinite  Reality  will  be  unavailable/impractical. 

♦  Samoff  continues  to  stonewall  the  PVR  Validation  Task,  shipping  us  useless  data 
and  avoiding  any  direct  contact.  We  have  defined  the  data  formats  and  exchange 
protocols  to  mitigate  their  issues,  and  have  shipped  them  the  first  two  DOQ  clips  (Ft. 
Dmm  site).  We'll  ship  the  last  4  DOQ  clips  for  LeJeune  and  APHill  when  they  verify 
their  ability  to  mn  on  the  first  shipment,  then  we'll  send  our  DPPDB  clips  for  all  four 
sites  (including  Fallon).  Expanded  Harris  team  to  help  calibrate  our  last  2  DOQ  clips 
and  clean  up  Samoff  s  calibration  data.  Examined  NG-CMS  survey  data  from  last 
year,  and  found  it  practically  worthless  for  absolute  error  characterization  (would  be 
lucky  to  evaluate  a  few  reference  images). 

♦  Finished  implementing  our  simple  &  rigorous  Predator  sensor  models  (2  models  are 
necessary  to  accommodate  the  various  Predator  configurations).  Arranging  travel  to 
General  Atomics  to  obtain  video  and  metadata  from  their  archives  that  meet 
calibration  and  validation  requirements.  Predator  ESD  decoder  HW  has  been 
procured,  and  have  begun  building  the  telemetry  decoder  software.  Will  validate  our 
Predator  PVR  processing  capability  when  we  get  good  data. 

♦  Prepared  slides  for  our  July  13th  IEEE  Video  Registration  Workshop  invited  talk, 
and  started  the  customer  &  Harris  approval  to  release  processes. 

♦  Prepared  3  orthomosaics  &  video  clips  for  the  Aug.  30th  AVS  Demo. 
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Accomplishments  for  the  Quarter  Ending  September  30. 2001 

PVR  Transition  and  Validation 

July  2001: 

♦  Hosted  on-site  visit  by  NIMA-TO  on  July  3rd. 

♦  Participated  in  the  NIMA-sponsored  Motion  Imagery  Geopositioning  Workshop  on 
24-25  July  to  share  our  AVS  experiences  with  the  Purdue  research  team. 

♦  Participated  in  a  TIM  with  General  D5mamics  -  Electronics  Systems  to  help  refine 
ideas  for  a  whitepaper  to  NIMA-NTA  to  deliver  video  georegistration  capability  that 
would  be  complimentary  to  their  MAAS  (Multimedia  Analysis  &  Archive  System) 
product.  We  owe  GD-ES  a  ROM  on  a  loose  integration  of  PVR  and  MAAS  (new 
API's).  Our  proposal  depends  on  the  Predator  sensor  model  validation  funded  by 
DARPA-AVS,  and  a  port  of  PVR  to  Origin  NT  funded  by  this  year's  Motion  Imagery 
IR&D. 

♦  Briefing  Harris  PVR  capabilities  and  AVS  experience  to  the  NMA  Precision 
Targeting  Tiger  Team  on  31  July  and  to  Navy  PMA281  on  6  Sept.  Arranging  a 
briefing  for  NIMA-GIT  (Geospatial  Info  Office,  Transnational  Center)  ASAP. 

♦  PVR  Validation  task  progress:  Harris  DOQ  runs  and  analysis  are  done.  Harris 
DPPDB  evaluation  is  in- work  and  should  be  complete  by  8/3/01 .  Our  outstanding 
results  have  been  presented  to  DARPA  and  NIMA.  Samoff  has  assigned  Jim  Matey 
to  run  their  PVR  system  on  our  clips,  and  we  are  making  good  progress  towards 
having  Samoff  validation  against  6  DOQ  clips  completed  in  time  for  the  August  30th 
AVS  Demo.  We'll  try  to  have  some  limited  Samoff  DPPDB  results  by  then  as  well. 

♦  The  final  AVS  Capabilities  Demonstration  in  Arlington,  VA,  is  set  for  August  30th. 
We  have  requested  DARPA  approval  to  deliver  our  AVS  demo  to  SGI's  Reality 
Center  in  Orlando  when  we  return  their  equipment.  The  increased  visibility  should  be 
beneficial  to  all  concerned. 

♦  Predator  telemetry  decoder  software  is  95%  complete.  We  need  to  engineer  a 
timestamp  solution  to  get  the  data  into  our  Telemetry  Database.  Predator  sensor 
model  validation  has  stalled  while  we  negotiate  General  Atomics  engineering  support 
to  help  get  our  hands  on  useful  data. 

August  2001: 

♦  Primary  focus  is  the  Final  AVS  Program  Capabilities  Demo  on  8/30  and  supporting 
DARPA/NIMA  TIM  on  8/29,  trip  report  to  follow  next  week.  Obtained  loaner  Origin 
3200  from  SGI  for  demo.  Validated  Harris  and  Samoff  PVR  Systems  using  6  DOQ 
clips,  validated  Harris  PVR  solution  using  4  DPPDB  clips  (Samoff  could  not  readily 
support  DPPDB);  results  to  be  presented  at  8/30  demo.  Predator  Sensor  Model 
Validation  stalled  due  to  lack  of  data  and  GA-ASI  cooperation. 

September  2001 : 

♦  Supported  DARPA/NIMA  TIM  on  8/29,  successful  AVS  Program  Capabilities  Demo 
on  8/30.  Obtained  loaner  Origin  3200  from  SGI  for  demo.  Showed  autonomous 
PVR  processing  of  streaming  video  data  without  interruption  for  over  30  minutes. 
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Potential  end-users  saw  dynamically  updating  situational  awareness  and  precision 
targeting  displays,  and  our  IR&D  developed  Georegistered  Video  Database  (GVDB) 
demo.  Performance  Validation  results  were  also  presented  (Harris  PVR  solution 
consistently  out-performs  Samoff  at  this  time). 

♦  Predator  Sensor  Model  Validation  stalled  due  to  lack  of  data  and  GA-ASI 
cooperation.  Upgrading  spare  GFE  Norpak  metadata  decoder  for  Predator  ground 
system  compatibility. 

♦  Samoff  slow  to  resume  work  on  validation  analysis,  we're  helping  them  identify  their 
problems  and  trying  to  accommodate  their  issues. 

♦  Expect  DARPA  AVS  program  to  extend  beyond  November  2001  due  to  NIMA 
support  and  continued  interest  in  additional  ACTD  study  efforts. 

Accomplishments  for  the  Quarter  Endin2  November  30. 2001 

PVR  Transition  and  Validation 

October  2001 : 

♦  Supported  the  NIMA  Motion  Imagery  Geolocation  Workshop  at  Wright-Patterson 
AFB  30  Oct  - 1  Nov.  Purdue  presented  their  error  modeling  work  needed  for  Smart 
Image,  DARPA  presented  AVS,  NIMA  briefed  plans  for  Smart  Image  and  Precision 
Engagement  programs,  for  which  Harris  has  submitted  ROMs.  Important  customer 
briefings  included  OSD  UAV  Roadmap  2000-2025,  AFRL/SNAR  geolocation 
programs  for  sensor  fusion,  and  geolocation  activities  for  the  ASC/RAB  Predator, 
PMA-263  VTUAV,  and  PEO  lEW&S  TUAV  programs. 

♦  Predator  Sensor  Model  Validation  stalled  due  to  lack  of  data  and  GA-ASI 
cooperation,  working  other  sources  of  data  with  NIMA.  Metadata  decoder  hardware 
came  back  from  the  Norpak  factory  with  upgraded  firmware  for  Predator  ground 
system  compatibility. 

♦  Samoff  slow  to  resume  work  on  validation  analysis,  we're  helping  them  identify  their 
problems  and  trying  to  accommodate  their  issues.  On  10/23  we  received  results  plots 
instead  of  the  processed  data  required  to  do  the  analysis  we're  under  contract  to 
perform. 

♦  There  are  discussions  of  another  round  of  field  demonstrations  in  December'Ol . 
Staffing  is  currently  down  to  approximately  1 .0  FTE. 

♦  Requested  no-cost  program  extension  to  the  contract  end  date  from  16  Nov  01  to  28 
Feb  02. 


November  2001: 

♦  Provided  an  AVS-PVR  capability  abstract  for  a  Lockheed  Martin  Mission  Specific 
Technology  Conference  in  support  of  USIGS-05. 

♦  Provided  a  demo  &  briefing  of  Harris  Motion  Imagery  capabilities  and  technology 
transition  plans  to  JWAC  at  a  TIM  on  1 1/15/01 . 
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♦  Provided  our  briefing  materials  &  demo  movies  for  video  georegistration  and 
RealSite  technologies  to  the  C2  Battlelab.  Expect  to  be  contacted  regarding  scope  of 
a  Jan'02  Video  ISR  initiative. 

♦  Supported  Harris  responses  to  the  Counter  Terrorism  BAA  dated  10/25.  Coordinated 
AVS  Program  inputs  and  wrote  up  a  Quad  Sheet  submission  for  AVS  Capabilities 
applied  to  Predator. 

♦  Supported  the  NMA  Motion  Imagery  Geolocation  Workshop  at  Wright-Patterson 
AFB  30  Oct  - 1  Nov.  Talked  with  Purdue  Profs.  Andrisani,  Bethel,  &  Mikhail  about 
our  DC  VO  1  paper. 

♦  Predator  Sensor  Model  Validation  stalled  due  to  lack  of  useful  data.  Reviewed  16 
hours  of  Predator  video  from  GA-ASI  and  the  UAV  Battle  Lab,  working  around 
problems  as  best  we  can.  Finished  the  new  software  interface  to  the  Predator 
metadata  decoder  hardware,  following  up  changes  rippling  thru  the  PVR  system. 
Have  captured  some  clips  for  initial  experimentation,  but  have  requested  better 
Baseline  Predator  GCS  data  from  GA-ASI. 

♦  In  response  to  an  invitation  from  UAV  Battle  Lab  to  bring  PVR  to  the  Predator  View 
demonstration  at  Indian  Springs,  NV,  we  will  participate  in  a  DARPA-sponsored 
Predator  UAV  PVR  Activity.  We  plan  to  get  the  best  possible  data  from  the  ACTD 
GCS  by  observing  and  influencing  Predator  operations  at  the  USAF  Weapon  School 
Mission  Employment  (ME)  Exercise. 

♦  Samoff  finally  provided  new  validation  data  on  1 1/27  (late),  which  we're  analyzing 
for  presentation  at  the  12/12/01  AVS  Program  Review  in  Annapolis,  MD. 

♦  AVS  contract  period  of  performance  is  now  extended  thru  2/28/02.  Remaining  work 
includes  finishing  the  Predator  model,  completing  our  validation  exercise,  writing  the 
final  report,  and  closing  out  the  program. 

Accomplishments  for  the  Quarter  Ending  February  28.  2002 

PVR  Transition  and  Validation 


December  2001  -  February  2002; 

♦  Supported  a  Predator  data  collection  exercise  with  the  1 1*  RS  training  group  at 
Indian  Springs,  NV,  4-7  Dec.  2001.  Obtained  good  video  data,  but  was  not  able  to 
obtain  suitable  reference  data  to  complete  model  calibration  and  PVR  validation  for 
operational  UAV  data  before  AVS  program  funding  ran  out. 

♦  Supported  the  Final  AVS  Program  Review  at  Annapolis,  MD,  12-13  Dec.  2001. 

♦  Wrote  and  submitted  the  PVR  Validation  Report  (HLDD  Update),  Final  Scientific 
and  Technical  Report,  and  Software  Product  End  Item  (AVS-PVR  Release  11) 
CDRLs. 
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